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ANALYSIS  REPORT 


This  document  completes  ths  report  of  ths  analysis  of  the  05C 
AIT  Course.  It  constitutes  a  supplement  to  the  Interim  Analysis 
Report  dated  25  Novsmber  1981. 

1.  CONTRACT  SPECIFICATIONS 

The  contract  (DABT60-81-C-0027)  specifies  that  the  analysis  report 
should  describe  the  "prerequisite  competency  skills  selected  for 
training. ”  It  states  that,  the  analysis  report  should  contain 
the  following  elements: 

1.  Prerequisite  competencies  selected  for  training 

2.  Prerequisite  competencies  not  selected  for  training 

3.  The  rationale  for  selection 

4.  The  relationship  of  prerequisite  skills  with  05C10 
course  learning  objectives 

5.  The  interrelationships  of  the  prerequisite  skills 

6.  The  results  of  the  verification  of  the  selected  skills 

7.  The  methodology  and  the  procedure  employed  in  performing 
the  analysis 

8.  The  principal  findings  and  any  difficulties  encountered 

Each  of  these  elements  will  now  be  discussed  with  reference 
being  made  to  the  Interim  Analysis  Report  if  the  information  is 
found  in  that  document.  Information  found  in  the  Interim  Analysis 
Report  will  not  be  repeated  here. 


Information  regarding  certain  elements  was  included  in  the  Design 
Report  of  18  December  1981.  When  this  occurs,  the  information 
will  be  summarised  here.  The  reader  can  refer  to  the  Design  Report 
for  a  more  detailed  discussion  of  methodology  and  findings. 

Before  discussing  each  of  the  elements  cited  above,  we  will  note 
and  discuss  changes  that  should  be  made  in  the  Interim  Analysis 
Report. 


The  outline  showing  the  recommended  structure  for  lesson  M-02* 
was  modified  as  shown  below: 


Original  version: 

M2. 8.1  Duplex  RT  must  have  minimum  1  MHz  or  10%  separation 
from  transmitting  frequency,  whichever  is  greater. 

*M2. 8.1.1  Find  10%  of  a  number  having  metric 
prefix 

*M2.8.1.2  Know  fact:  purpose  is  to  prevent 
overlap  of  signals 


Revised  version: 

M2. 8.1  Determine  the  frequency  for  the  duplex  RT 

*M2 .8.1.1  Obtain  the  duplex  frequency  from 
the  CEOX 

M2. 8. 1.2  Know  fact:  Duplex  RT  must  have  a 
minimum  of  1  MHz  or  10%  separation 
from  the  transmitting  frequency, 
whichever  is  greater. 

*M2.8.1.2.1  Find  10%  of  the  trams- 
mi t ting  frequency 

*M2.8.1.2.2  Compare  10%  of  a  frequency 
with  the  number  1 

*M2 . 8 . 1 . 2 . 3  Find  the  frequencies 

which  are  10%  above  and 
10%  below  the  transmitting 
frequency 


^Install,  adjust,  transmit  and  receive  messages  using  Radio 
Teletypewriter  Set  AN/GRC-142  and  AN/GRC-122  in  normal  remote 
and  duplex  operation. 


*M2 . 8 . 1 . 2 . 4  Compare  the  duplex 

frequency  obtained  from 
the  CEO I  with  the  frequencies 
which  are  10%  above  and 
10%  below  the  transmitting 
frequency 

*M2 .  8 . 1 . 2 . 5  Know  fact:  If  the  duplex 
frequency  from  the  CEOI 
does  not  meet  the  minimum 
separation,  notify  your 
supervisor 

*M2 . 8 . 1 . 2 . 6  Know  fact:  The  purpose 

of  the  minimum  separation 
is  to  prevent  the  overlap 
of  signals 


The  need  for  the  change  was  prompted  by  the  receipt  of  more 
detailed  information  from  subject  matter  experts  at  the  signal 
School.  The  effect  of  the  change,  as  far  as  prerequisite 
competencies  are  concerned,  is  to  add  two  basic  skill  procedures 
to  the  list  of  candidates  for  the  FBSEP ,  items  M2. 8. 1.2.2  and 
M2. 8. 1.2. 4.  The  other  changes  have  implication  for  the  05C  AIT 
Course  but  not  for  the  FBSEP. 


Other  Chances 

The  fourth  sentence  of  the  second  paragraph  on  page  4  should 
read  as  follows:  "Some  examples  of  prerequisite  skill  procedures 


3.  PREREQUISITE  COMPETENCIES  SELECTED  FOR  TRAINING 

The  selection  of  prerequisite  competencies  for  training  requires 
three  operations.  First,  on  the  basis  of  an  analysis  of  the 
content  of  the  05C  AIT  Course  lessons,  prepare  a  tentative  list 
of  knowledge  and  skills  which  may  constitute  prerequisites  for 
specific  05C  AIT  Course  lessons.  These  first  level  prerequisites 
are  sometimes  called  "component,”  (Okey,  1973).  A  description 
of  the  knowledges  and  skills  so  identified,  the  rationale  for 
selection,  and  the  methodology  and  procedure  followed,  are  con¬ 
tained  in  the  Interim  Analysis  Report. 

Second,  analyze  the  prerequisite  knowledges  and  skills  derived 
in  step  #1  and  identify  knowledge  and  skills  which  may  constitute 
prerequisites  to  the  knowledge  and  skills  derived  in  step  #1. 
Continue  this  process  of  developing  successively  lower  levels 
of  prerequisites  until  a  point  is  reached  where  it  is  felt  that 
most  students  will  already  possess  the  knowledge  and  skills 
required  to  perform  the  next  higher  task.  The  result  of  this 
process  is  a  learning  hierarchy,  a  description  of  the  learning 
hierarchies  so  developed,  the  knowledge  and  skills  comprising 
each  hierarchy,  the  rationale,  and  the  methodology  and  procedure 
followed,  are  contained  in  the  Design  Report. 

Third,  cluster  individual  knowledge  and  skills  (those  not  part 
of  a  hierarchy)  and  learning  hierarchies  into  modules.  The 
prerequisites  identified  in  steps  «1  \nd  #2  were  clustered  into 


four  nodules,  study  skills,  language,  reading,  and  math.  The 
specific  knowledge,  skills  and  learning  hierarchies  assigned 
to  each  nodule  are  found  in  the  Design  Report. 

Fourth,  determine  if  the  knowledge,  skills,  and  learning  hier¬ 
archies  developed  by  logical  analysis  in  steps  #1  and  #2  are 
valid  and  are  supported  by  data.  This  is  accomplished  by 
developing  tests  to  measure  each  prerequisite  knowledge  and 
skill,  administering  these  tests  to  students  enrolled  in  the 
05C  AIT  Course,  collecting  students'  scores  on  05C  AIT  lesson 
tests,  analyzing  student  scores  on  individual  tests  and  deter¬ 
mining  the  relationship  between  test  scores. 

The  analysis  of  student  scores  on  the  test  used  to  measure  pre¬ 
requisite  knowledge  and  skills  affords  an  opportunity  to  verify 
the  etention  (or  exclusion)  of  the  knowledge  and  skills  tenta¬ 
tively  selected  for  inclusion  in  steps  #1  and  #2.  If  a  signifi¬ 
cant  number  of  students  representative  of  the  target  population 
do  poorly  on  items  designed  to  test  a  particular  knowledge  or 
skill,  that  is  evidence  for  retaining  the  knowledge  or  skill  in 
the  FBSEP.  If,  on  the  other  hand,  a  significant  number  of 
students  representative  of  the  target  population  do  well  on 
items  designed  to  test  a  particular  knowledge  or  skill,  that 
is  evidence  for  dropping  the  knowledge  or  skill  from  the  FBSEP. 


Test  items  were  prepared  for  all  of  the  knowledge  and  skills 
identified  in  steps  #1  and  #2  and  the  test  was  administered  to 
a  sample  of  students  prior  to  their  receiving  any  instruction 
in  the  05 C  AIT  Course.  A  criterion  of  95%  correct  was  established 
because  the  students  to  which  the  test  was  administered  had 
higher  aptitude  scores  than  the  target  population.  If  95%  or 
more  of  the  students  answered  questions  related  to  a  particular 
knowledge  or  skill  correctly/  that  knowledge  or  skill  was  deleted 
from  the  list.  Conversely/  if  less  than  95%  of  the  students 
answered  the  questions  correctly  the  knowledge  or  skill  was 
retained. 

Using  the  95%  criterion,  only  three  prerequisites  were  dropped 
from  the  list  of  prerequisites  tentatively  selected  for  inclusion 
in  the  FBSEP .  The  three,  all  math  skills,  were: 

Deciding  if  a  number  is  greater  than  1 

Comparing  the  value  of  two  numbers 

Deciding  if  a  number  is  between  two  other  numbers 

The  corresponding  FBSEP  lessons  were,  consequently,  dropped  from 
the  roster.  Since  these  skills,  none  the  less,  are  prerequisite 
for  an  AIT  task,  a  brief  explanation  of  them  was  incorporated 
in  another  FBSEP  lesson.  Included  is  their  connection  with  the 
appropriate  AIT  task  and  with  other  related  FBSEP  math  skills. 
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4.  THE  RELATIONSHIP  OF  PREREQUISITE  SKILLS 
WITH  05C10  LEARNING  OBJECTIVES 

The  relationship  of  prerequisite  skills  with  05C10  learning 
objectives  can  be  looked  at  in  three  ways.  The  first  way  is 
in  terms  of  logical  analysis.  The  analyst,  after  examining 
lesson  materials  and  interviewing  the  faculty,  hypothesizes 
that  certain  knowledge  and  skills  constitute  prerequisites  for 
a  particular  lesson.  The  second  way  is  in  terms  of  statistical 
relationship.  The  analyst  prepares  tests  to  measure  the  inci- 
dence  of  the  prerequisite  knowledge  and  skills  and  relates 
student's  scores  on  these  tests  to  student's  scores  on  lesson 
tests.  The  third  way  is  in  terms  of  practical  effect.  The 
analyst  prepares  training  materials  to  correct  deficiencies  in 
prerequisite  knowledge  and  skills  and  then  determines  the  im¬ 
pact  that  such  remedial  training  has  on  lesson  tests. 

The  results  of  the  first  approach  is  described  in  this  section 
of  the  Analysis  Report.  The  results  of  the  second  approach  will 
be  described  in  the  section  of  this  report  titled  Results  of  the 
Verification  of  Basic  Skills.  The  results  of  the  third  approach 
will  be  examined  at  a  later  point  in  time,  after  the  basic  skills 
lesson  have  been  implemented.  This  will  take  place  some  time 
after  the  present  contract  is  completed. 

A  discussion  and  table  describing  the  relationship  between  pre¬ 
requisite  knowledges  and  skills  with  05C10* lessons  appears  in 
the  Design  Report.  It  should  be  noted  that  there  have  been 


minor  changes  in  the  prerequisites  since  the  publication  of  the 
Design  Report  and  that  the  relationships  shown  in  the  Design 
Report  are  based  on  logical  analysis  rather  than  statistical 
data.  Table  €  of  the  Development  Report  shows  the  updated  re¬ 
lationship  between  FBS2P  lessons  and  05C  AIT  lessons. 

Table  1,  based  on  inforaation  in  the  Design  Report,  shows  the 
distribution,  by  OSC  AIT  lesson,  of  those  prerequisite  knowledges 
and  skills  which  are  lesson  specific,  that  is,  excluding  those 
prerequisites,  such  as  reading,  which  apply  to  all  lessons.  For 
examples,  OSC  AIT  Lessons  H-02  and  M-02  each  had  12  prerequisites. 
Lesson  L-01,  on  the  other  hand,  had  no  prerequisites. 
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NUMBER  OF 
PREREQUISITES 


05C  AIT  LESSON 


12 

11 

10 

9 

8 

7 

6 

5 

4 

3 

2 

1 

0 


H-02  and  M-02 
N-02 

H-01  and  J-01 
J-02 

K-01  and  M-03 

J-01,  J-03,  N-01,  N-03,  and  N-04 
H-05  and  N-OS 
X-02  and  N-06 
H-03 

H-04 ,  H-06,  and  M-01 
F-02 

L-02  and  L-03 
L-01 


Table  2,  also  based  on  data  in  the  Design  Report,  shows  the 
distribution  of  OSC  AIT  lessons  by  lesson  specific  prerequisites. 
As  can  be  seen  in  the  table,  the  number  of  lessons  supported 
by  a  prerequisite  ranged  from  14  to  1.  The  most  frequent  pre¬ 
requisites  are  the  following. 


NOTE:  Prerequisite  identification  numbers  and  titles  shown  are 
not  those  in  Design  Report,  but  those  in  current  use. 


B-33* 

Match  taxt  with  equipment 

14 

H 

a 

a 

• 

8 

m 

B-08 

Find  infornation  in 
illustration 

12  lasaons 

B-31 

Find  information  in 
illustration  using 
ralatad  taxt 

12 

lassons 

B-32 

Identify  parts  of  aquipnant 
using  labeled  illustration 

12 

lassons 

B-33 

Perform  a  procadura  using 
ralatad  illustration /taxt 

12 

lassons 

B-09 

Find  information  in  tablas 

11 

lassons 

TABLE  2 


DISTRIBUTION  OF  05C  AIT  LESSONS  BY 
LESSON  SPECIFIC  PREREQUISITES 


NUMBER  OF  05C  AIT 
LESSONS 


14 

12 

11 

9 

7 

6 

5 

4 

3 

2 


B-09* 

B-08,  B-31 ,  B-32,  B-33 

B-09 

A- 09 

B-07 

C-01 

C-05/C-07 

C-09 

A- 06,  A-07 ,  A-08,  A-09, 

B-04 ,  B-05 ,  B-10,  B-28,  B-29,  B-30, 
C-02,  C-03 ,  C-04 

B-03,  B-05,  B-ll ,  C-07,  C-10,  D-01 
D-02,  D-03,  D-04 ,  D-05,  D-06,  D-06  , 
D-07,  D-07*,  D-08,  D-09,  D-10,  D-12, 
D-13,  D-14 


*This  praraquisits  was  listad  in  tha  Design  Report  as  B-09.  It 
is  now  incorporated  in  B-33,  Perforsdng  a  Procadura  Using 
II lustration /Text . 
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j 

5.  THE  INTERRELATIONSHIPS  OF  THE  BASIC  SKILLS 

The  interrelationship  of  basic  skills  can  often  be  portrayed  by  ^ 

learning  hierarchies  in  which  tasks  are  arranged  to  show  a  pre¬ 
requisite  relationship.  Immediately  beneath  each  task  and  *  > 

connected  to  it  with  a  line,  the  analyst  shows  those  tasks  which  « 

are  prerequisites,  that  is,  the  knowledge  and/or  skills  which 
individuals  must  have  in  their  repertoire  in  order  to  perform 

■ 

the  task  immediately  above  in  the  learning  hierarchy.  The  pro-  ► 

*s 

cedure  normally  is  continued  until  the  analyst  has  identified  ; 

all  knowledge  and  skills  on  which  the  target  population  is 

suspected  to  be  deficient.  j 

Theory 

The  first  set  of  learning  hierarchies  are  based  on  the  judge-  j 

ments  of  the  analyst  who  examines  the  terminal  task  to  determine 
its  prerequisite  and  then  analyses  each  prerequisite  (enabling) 
task  to  determine  their  prerequisites.  Invariably  these  learn-  j 

ing  hierarchies  are  revised.  Some  revisions  are  based  on  the 
judgement  of  lesson  writer  and  test  writers  who  realize  that 
certain  necessary  knowledge  and  skills  have  been  overlooked  or 
that  other  knowledge  and  skills  are  superfluous.  Other  revisions 
are  based  on  data  secured  by  administering  tests  designed  to 
measure  the  presence  or  absence  of  the  knowledge  and  skills  in 
the  target  population.  These  tests  may  precede  or  follow  train¬ 
ing  on  the  knowledge  and  skills  in  the  learning  hierarchy. 
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The  advisability  of  including  or  excluding  a  particular  knowledge 
or  skill  in  a  learning  hierarchy  can  be  looked  at  in  an  absolute 
sense  or  relative  to  the  task  for  which  it  may  be  a  prerequisite. 
If  the  test  scores  indicate  that  many  students  have  the  knowledge 
or  skill  under  consideration,  there  is  little  merit  in  includ¬ 
ing  it  in  the  learning  hierarchy.  If,  on  the  other  hand,  many 
students  do  not  have  the  knowledge  or  skill,  it  probably  should 
be  included  in  the  learning  hierarchy.  Low  scores  also  should 
prompt  the  analyst  to  carefully  examine  the  knowledge  or  skill 
to  determine  if  it  has  prerequisites,  the  mastery  of  which  would 
facilitate  performance  on  the  first  knowledge  or  skill.  The 
criterion  of  what  constitutes  "have”  or  "do  not  have,"  is  a 
matter  of  judgement. 

The  interrelation  of  knowledge  and  skills,  as  shown  by  test 
scores,  also  can  be  used  as  a  basis  for  inclusion  or  exclusion. 

If  a  true  prerequisite  relationship  axists,  test  scores  should 
show  a  positive  relationship  between  a  knowledge  or  skill  and  its 
prerequisites.  Scores  on  a  prerequisite  knowledge  or  skill 
should  be  equal  or  higher  than  scores  on  the  knowledge  or  skill 
for  which  the  first  knowledge  or  skill  is  a  prerequisite.  There 
should  be  no  instances  where  scores  on  a  prerequisite  knowledge 
or  skill  are  lower  than  the  scores  on  the  knowledge  or  skill  for 
which  it  is  a  prerequisite. 


If  a  positive  relationship  exists  between  two  knowledge/skills , 
this  is  an  argument  for  their  retention  and  for  the  hypothesized 
prerequisite  relationship.  If  there  is  no  relationship  between 
the  two  knowledge /skills ,  this  is  an  argument  for  exclusion. 

If  the  relationship  is  negative,  this  is  an  argument  for  revers¬ 
ing  their  relationship. 

Two  cautions  should  be  kept  in  mind  when  using  test  scores  as 
a  basis  for  identifying  prerequisites  and  establishing  learning 
hierarchies.  First,  test  development  should  follow  established 
test  development  procedures.  Second,  care  should  be  taken  to 
insure  that  test  will  be  sensitive  to  whatever  prerequisite 
relationship  does  exist  between  the  knowledge  and  skills  under 
consideration.  If  care  is  not  taken,  the  prerequisite  relation¬ 
ship  may  be  masked  or  overlooked.  An  example  of  each,  based 
on  our  experience  in  FBSEP  may  make  these  points  clear. 

Let  us  assume  that  the  terminal  task  is  to  fill-out  a  form  and 
that  one  of  the  prerequisites  is  "spelling  military  terms." 

Care  must  be  taken  to  insure  that  the  test  that  measures  "filling 
out  a  form"  considers  the  spelling  of  military  terms  on  the  form. 
If  the  test  only  considers  that  all  entries  are  made  and  in  the 
proper  box  on  the  form,  the  effect  of  the  prerequisite  will 
have  been  overlooked.  Of  course,  the  assumed  prerequisite  may 
not,  in  fact,  be  related  to  the  high-order  skill. 


Critical  factors  cannot  ba  overlooked  in  the  area  of  math  and 
this  may  account  for  the  frequent  finding  that  validated 
learning  hierarchies  are  more  evident  in  math  than  in  other 
intellectual  skills (Briggs,  1968  as  cited  by  Okey,  1973),  y0r 
example  two  prerequisites  for  long  division  are  multiplication  and 
subtraction  and  it  is  not  possible  to  solve  a  long  division  problem 
without  multiplying  and  subtracting  numbers. 

Here  is  an  example  where  the  prerequisite  effect  is  masked. 

Let  us  assume  that  the  terminal  task  is  to  find  information  in 
a  publication  and  that  two  of  the  prerequisites  are  "using  an 
index”  and  "using  a  table  of  contents."  If  the  publication  has 
but  a  few  pages,  the  student  can  find  the  correct  page  by  leaf¬ 
ing  through  the  document  rather  than  by  using  the  table  of 
contents  or  index.  Admittedly ,  the  "leafing  through”  procedure 
would  probably  take  more  time  but  the  test  would  be  sensitive 
to  this  element  only  if  time  was  taken  into  account  when  scoring 
this  test  item.  A  time  limit  for  the  test  as  a  whole,  assuming 
that  it  includes  a  variety  of  test  items,  would  not  be  an  adequate 
control  of  the  time  factor  which  would  be  critical  for  this  type 
of  a  prerequisite  relationship. 

Failure  to  take  ."time”  into  account  can  also  mask  a  prerequisite 
relationship  in  math.  Let  us  assume  that  the  terminal  task  is 
to  solve  a  problem  and  then  find  10%  of  the  answer,  and  that 
one  of  the  prerequisites  is  to  find  10%  of  a  number  by  moving 
the  decimal  point.  Ifce  student  can  find  10%  of  the  answer  by 
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the  laborious  multiplication  procedure  rather  than  by  moving 
the  decimal  point.  Multiplication  would  take  more  time  (and 
would  be  more  prone  to  errors ) ,  but  the  test  would  not  be  sensi 
tive  to  this  fact  unless  the  original  problem  was  timed. 

FBSEP 

The  first  set  of  learning  hierarchies  for  FBSEP  basic  skills 
appears  in  the  Design  Report.  The  basic  skills  were  first  put 
into  four  clusters  (language,  reading,  study,  and  math)  and 
then  the  skills  within  each  cluster  were  arranged  into  learning 
hierarchies  on  the  basis  of  the  judgement  of  the  analyst.  In 
the  course  of  preparing  test  items  for  a  diagnotic  test  to 
measure  the  incidence  of  the  knowledge  and  skills  in  the  target 
population,  some  changes  were  made  in  the  learning  hierarchies 
included  in  the  Design  Report.  These  changes  included  adding 
a  knowledge/skill,  deleting  a  knowledge/ skill,  and  changing 
the  location  in  a  learning  hierarchy  of  a  knowledge/skill. 

The  Diagnostic  Test  was  administered  in  the  Verification  run. 
The  responses  of  105  students  in  the  Verification  run  were 
used  to  validate  20  of  the  hypothesized  prerequisite  relation¬ 
ships  shown  in  the  revised  learning  hierarchies.  Of  the  twenty 
pairs  of  basic  skills,  eight  dealt  with  reading  skills,  three 
with  language  skills,  and  nine  with  math  skills.  The  data  was 
analyzed  by  Horst  (1982)  using  Chi  Squares  to  test  the  signifi¬ 
cance  of  the  relationship. 


Zf  a  "true"  hierarchical  relationship  exists,  student  perform 
nance  on  a  prerequisite  skill  should  be  higher  than  on  the 
higher  order  basic  skill  that  it  supports.  This  pattern  was 
evident  on  two  of  the  eight  reading  skill  relationships,  on  two 
of  the  three  language  skill  relationships,  and  on  all  nine  of 
the  math  skill  relationships. 

Of  the  thirteen  relationships  in  which  student  performance  on 
a  prerequisite  skill  was  higher  than  on  the  skill  for  which 
it  was  a  prerequisite,  only  three  were  statistically  signifi¬ 
cant  at  the  .05  level  or  better.  The  three, all  math  skills, 
are  shown  below. 

Add  or  subtract  hours  moving  across  days 

Add  hours  to  military  time 

Find  the  numbers  which  are  10%  above  and  lot  below 
a  given  number 

Find  10%  of  a  number 

Divide  the  number  468  by  a  number  containg  a 
decimal  and  round  off  the  answer  to  the  nearest 
tenth. 

Round  off  a  number  to  the  nearest  tenth. 

A  number  of  factors  may  account  for  the  lack  of  statistical 
support  for  the  hypothesized  hierarchical  relationships.  First, 
the  number  of  test  items  was  limited  because  only  two  hours 
were  allocated  for  test  administration.  Of  the  30  basic  skills 
used  in  the  analysis,  ten  were  tested  with  but  one  item,  six 


with  but  two  items,  and  eight  with  three  items.  The  Diagnostic 
Test  can  be  considered  as  a  sample  of  the  items  that  test  the 
basic  skills.  For  a  more  accurate  test  of  the  hierarchy,  a  more 
complete  set  of  items  should  be  used.  For  example,  the  hypothe¬ 
sized  relationship  between  "find  information  in  text”  and  "find 
information  in  table  of  contents"  was  not  supported  by  data. 

The  likely  reason  is  that  the  "text”  was  not  a  publication  but 
one  and  one-half  pages  from  a  publication.  A  table  of  contents 
would  have  been  of  little  assistance  in  this  situation.  Second, 
in  some  cases,  the  test  items  used  to  measure  a  basic  skill  were 
very  easy.  The  mean  percent  correct  on  thirteen  of  the  thirty 
basic  skills  used  in  the  analysis  was  80«  or  better,  in  one 
instance  reaching  99%.  (In  the  Diognostic  Test  as  a  whole  only 
18  of  the  100  multiple-choice  items  for  a  sample  of  325  students 
showed  more  than  80*  scoring  correctly.)  The  easy  test  items 
were  retained  because  it  was  felt  that  the  105  students  used  in 
the  analysis  had  higher  aptitude  scores  than  will  be  true  of 
students  who  are  the  eventual  target  of  the  FBSEP . 

Third,  reading  skills,  as  has  been  noted  earlier,  generally  do 
not  form  hierarchical  relationships.  Although  in  theory  it  would 
seem  that  knowing  vocabulary  would  facilitate  reading  compre¬ 
hension,  it  is  also  true  that  a  reading  passage  provides  con¬ 
text  which  facilitates  comprehension  even  if  some  vocabulary  is 
unfamiliar.  This  may  account,  for  example,  for  the  lack  of  a 
statistical  relationship  between  "comprehend  reading  passage,” 
and  "match  terms  and  their  definition."  It  should  be  noted 


6.  CHANGES  IN  THE  LEARNING  HIERARCHIES 


As  has  been  noted  earlier,  the  learning  hierarchies  underwent  a 
number  of  revisions  in  the  course  of  the  research  and  development 
effort.  This  section  of  the  report  will  describe  some  of  the 
revisions  that  were  made  and  the  rationale  for  the  changes. 

Language  Skills 

In  the  Design  Report,  "spell  commonly  used  military  terms"  was 
shown  as  a  prerequisite  for  "fill-out  forms"  and  "write  text  of 
message  presented  orally."  It's  prerequisite  relationship  with 
"fill-out  form"  was  deleted  because  in  the  test  for  "fill-out 
forms,"  the  student  is  given  printed  copy  which  he  then  trans¬ 
fers  to  (enters  on)  blocks  on  the  form.  The  ability  to  "spell" 
is  not  required. 

Reading  Skills 

In  the  Design  Report,  "match  terms  and  their  definitions,"  and 
"integrating  information  to  form  concepts,"  are  both  shown  as 
prerequisite  to  "comprehend  reading  passages . "  "Match  terms 
and  their  definitions"  was  removed  from  this  learning  hierarchy 
because  it  was  felt  that  the  general  context  of  the  reading 
passage  would  outweigh  the  effect  of  knowledge  of  terms  and 
their  definitions  in  having  an  impact  on  comprehension. 


Note:  There  have  been  minor  changes  in  the  wording  of  basic 
skill  titles  and  major  changes  in  the  identification 
letters /numbers  assigned  to  basic  skills  between  the 
Design  Report  and  this  Analysis  Report. 


Study  Skills 

In  the  Design  Report,  eight  basic  skills  were  arranged  into  a 
learning  hierarchy  as  shown  below.  It  was  acknowledged,  at  the 
tine,  that  it  was  not  a  "true”  learning  hierarchy,  but  rather  a 
convenient  way  to  group  skills  that  are  clearly  related  to  one 
another.  There  was  no  intention  of  conducting  training  at  C-03 
C-07,  or  C-08.  At  these  points  in  the  hierarchy,  the  intent 
was  to  test  only. 


C-08 

UNDERSTAND  THE 
INSTRUCTIONAL 
SYSTEM 


C-03  C-04  C-07 

WITHIN  A  TRAINING  AIDS  ACROSS 

LESSON  AND  DEVICES  LESSONS 


C-01 

RANTS  AND 
FUNCTIONS  Of 
STUDY  GUIDE 


C-02 

SEQUENCE 
Of  EVENTS 
WITHIN 
A  LESSON 


C-05 

PARTS  AND 
FUNCTIONS 
ACROSS 
LESSONS 


C-06 
SEQUENCE 
OF  EVENTS 
ACROSS 
LESSONS 


The  above  hierarchy  was  simplified  into  the  one  shown  on  the 
next  page  when  it  became  apparent  that  skills  C-01  and  C-02 
could  conveniently  be  combined  into  one  lesson  and  that  skills 
C-05  and  C-06  also  could  conveniently  be  combined  into  one  lesson. 
On  the  other  hand,  "Understand  FBSEP”  was  inserted  in  the  learn¬ 
ing  hierarchy  because  it  was  felt  that  students  needed  an  orienta¬ 
tion  to  the  remedial  program  in  which  they  were  participating. 
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It  should  Also  be  noted  that  at  one  stage  during  lesson  develop¬ 
ment,  there  were  three  additional  math  lessons  dealing  with  the 
comparison  of  the  value  of  numbers.  As  explained  in  the  Develop 
■ent  Report,  further  analysis  showed  these  skills  were  not  func¬ 
tional  and,  consequently,  these  lessons  were  dropped  from  the 
FBSEP  schedule.  The  elimination  of  these  lessons,  however,  did 
not  alter  the  hierarchal  order  of  the  remaining  math  lessons. 

A  copy  of  each  of  the  four  revised  annex  maps  follows.  Each 
map  reflects  how  the  lessons  within  that  annex  are  arranged  in 
hierarchal  order,  demonstrating  their  interrelationship  with 
enabling  lessons  and  with  higher  level  lessons. 


Pigure  1.  Learning  Hierarchy,  Annex 


Figure  2.  Learning  Hierarchy,  Annex 


Learning  Hierarchy,  Annex  C 


7.  INCIDENTAL  ACQUISITION  AND  RETENTION 
OF  BASIC  SKILLS 

On«  of  the  major  goals  of  the  FBSEP  effort  was  to  identify 
basic  knowledge  and  skills  which  constitute  prerequisites 
for  the  technical  knowledge  and  skills  taught  in  the  05C 
AIT  Course.  The  ultimate  goal  was  to  design  a  training 
program  to  impart  these  basic  knowledge  and  skills  to  those 
students  who  may  lack  them,  thus  facilitating  their 
progress  through  the  05C  AIT  Course. 

If  our  initial  identification  of  basic  knowledge  and  skills 
is  correct,  if  one  must  possess  them  in  order  to  complete 
the  course,  then  we  must  assume  that  students  who  complete 
the  course  have  the  prerequisite  knowledge  and  skills. 

Prior  to  the  implementation  of  FBSEP,  the  student  could 
have  acquired  the  prerequisite  knowledge  and  skills  in  one 
of  two  ways.  The  student  either  brought  them  with  him 
when  he  started  the  05C  AIT  Course  or  he  acquired  them 
incidentally  while  undergoing  training.  In  either  case, 
the  student  would  be  expected  to  use  the  knowledge  or  skill 
in  order  to  perform  the  technical  tasks  which  constitute 
the  05C  AIT  Course.  This  "use”  constitutes  "practice” 
of  the  particular  knowledge  and  skills.  If  our  reasoning 
is  correct  student's  scores  on  prerequisite  knowledge  and 
skills  at  the  end  of  training  (after  incidental  learning 
and  required  practice)  should  be  higher  than  at  the  start  of 
training  (prior  to  incidental  learning  and  required  practice) 


This  hypothesised  effect  was  measured  by  administering  the 
Diagnostic  Test  twice  to  47  students,  first  before  they  started 
the  05C  AZT  Course  and  again  after  they  had  graduated  from  the 
05C  AZT  Course.  The  data  is  presented  in  Table  3.  The  text  is 
shown  for  those  test  items  in  which  the  difference  between  be¬ 
fore  and  after  scores  was  15%  or  greater. 


TEST 


PERCENT  CORRECT 
BEFORE  AFTER 

TRAINING  TRAINING 


DIFFERENCE 


TABLE  3  (cont'd.) 


PERCENT 

CORRECT 

BEFORE 

AFTER 

TEST  ITEM 

TRAINING 

TRAINING  DIFFERENCE 

Math  Skills  (cont'd) 


Beading  Skills 

43  Fill-in  gaps  in  outline 
format 

44 

45 


52  Understand  paragraph  number¬ 
ing  system 

53  Use  index  in  CEO I 

54 

55  Use  table  of  contents  in 
a  TM 


67  Find  information  in  an 
illustration 


TABLE  3  (cont'd) 


TEST  ITEM 


PERCENT  CORRECT 
BEFORE  AFTER 

TRAINING  TRAINING  DIFFERENCE 


Reading  Skills 


71 

72 

73  Find  information  in  trouble¬ 
shooting  table 

74 

75  Find  information  in  diagram 

76 

77 

78 

79 

80 


81 

82 

83 

84 

85  Identify  definition  of 
diameter 

86 

87  Identify  definition  of 
precedence 

88  Identify  definition  of 
deflection 

89  Find  information  in  text 
re  suffix 

90  Find  information  in  text 
re  suffix 


91 

92 

93 

94 

95 

96 

97  Find  error  in  a  message 


65 

72 

68 

78 

65 

87 

78 

93 

55 

70 

72 

76 

80 

82 

87 

74 

74 

85 

46 

53 

29 

38 

61 

59 

51 

55 

36 

38 

38 

53 

55 

61 

55 

80 

59 

74 

61 

89 

70 

95 

80 

91 

72 

80 

87 

89 

63 

76 

40 

44 

38 

51 

31 

47 

Improvement  was  shown  on  63  of  the  92  test  items  for  which  data 
was  available,  though  most  of  the  differences  are  below  15%.  A 
possible  explanation  of  why  more  improvement  is  not  evident  is 
the  "ceiling  effect ,  "  that  is ,  students  who  did  well  on  the  test 
before  training  had  little  room  for  improvement.  Of  the  17  items 
in  which  differences  were  15%  of  greater,  only  one  (#53)  was  in 
the  unexpected  direction;  that  is,  students  performed  better  be¬ 
fore  training  than  after  training.  We  have  no  explanation  for 
this  reversal  because  students  showed  modest  improvement  on  item 
#54  which  posed  a  similar  task. 

An  alternative  explanation  of  the  gains  shown  could  be  student 
familiarity  with  the  test  item  because  the  same  test  items  were 
used  for  both  administrations.  This  seems  unlikely  since  students 
received  no  feedback  after  the  first  administration,  and  approxi¬ 
mately  11  weeks  separated  the  two  administrations  of  the 
Diagnostic  Test. 

Table  4  presents  the  same  data,  this  time  with  test  items 
arranged  to  show  the  FBSEP  lesson  which  they  support.  For 
example,  teat  items  43  and  44  support  FBSEP  lesson  B-03. 


Mo.  of 
Ifeadonca 

Mo.  of  Docroaalag  t  Utia 

Diag.  Tut  Nma  Moan  feudonta  (cor*  or  oa 

Uwu  Xtaa  or  Me.  of  Tnuit  roituit  Xneroaaing  M—  lining  Soloqtad 

Mo.  CXoatara  goa*  ticca  Sear*  Seer*  (cor*  Umm  (air* 


Lessons  B-22  through  B-27  are  lessons  in  Comprehending  Reading 
Passages  on  Topics  Related  to  Radio  Teletype  Communications. 

The  increase  in  scores  may  be  attributed  to  the  fact  that  the 
students  acquired  much  of  the  information  contained  in  the 
questions  during  the  course.  They  may  have  been  able  to  answer 
the  questions  without  reading  the  passage.  The  improvement  in 
their  scores  would,  therefore,  reflect  an  increase  in  knowledge 
and  not  an  improvement  in  reading  comprehension  skill. 

Table  5  presents  Diagnostic  Test  scores  by  the  three  major  skill 
areas,  before  starting  and  after  completing  05C  AIT  training, 
before  the  onset  of  the  FBSEP.  The  results  show  a  significant 
improvement  in  basic  skills  knowledge  for  the  total  test  as  well 
as  for  the  language  and  reading  subtests.  No  change  is  evident 
for  math.  Students  who  graduate  acquire  or  retain  the  basic 
knowledge  and  skills  which  are  prerequisite  for  learning  the 
technical  tasks  taught  in  the  05C  AIT  Course. 

It  is  interesting  to  compare  the  mean  score  obtained  by  the  course 
graduates  with  the  highest  possible  scores.  Noting  that  the  test 
scores  reported  here  are  means,  it  appears  that  there  is  room  for 
improvement  here. 


Language  Skills 

22 

14.7 

17.9 

6.28* 

Math  Skills 

«■  28 

21.0 

21.7 

1.65 

Reading  Skills 

70 

49.2 

52.8 

3.02* 

Total 

120 

84.9 

92.5 

4.89* 

*  ■  Significant  at 

.01  level 

or  better  . 

Two  points  should  be  siade  before  ending  this  section  of  the 
report.  First  while  the  data  shows  that  course  graduates 
acquire  and  retain  necessary  basic  skills  without  the  FBSEP , 
it  is  likely  that  the  presence  of  the  FBSEP  would  facilitate 
their  progress.  The  FBSEP  should  make  it  possible  for  students 
to  experience  less  problems  and  move  more  rapidly  through  the 
05C  AIT  Course.  Second,  we  have  no  comparable  acquisition  and 
retention  data  on  students  who  failed  to  graduate  from  the  05C 
AIT  Course.  It  is  likely,  however,  that  their  failure  was  due, 
at  least  in  part,  to  their  failure  to  acquire  the  basic  knowledge 
and  skills  which  are  prerequisite  for  the  05C  AIT  Course. 


8.  VERIFICATION  OF  OBSERVED  DIFFICULTIES 
IN  05C  AIT  COURSE 

Soon  after  the  start  of  the  FBSEP  project,  members  of  the 
research  team  conducted  systematic  observations  of  training  and 
testing  in  the  05C  AIT  Course.  One  of  their  findings  was  that 
some  of  the  knowledge  and  skills  taught  in  the  course  were 
tested  inadequately  or  were  not  tested  at  all.  One  possible 
effect  of  this  weakness,  is  that  same  students  may  not  possess 
technical  knowledge  and  skills  which  are  prerequisite  to  other 
knowledge  and  skills  taught  in  the  course. 

In  order  to  verify  the  hypothesis  that  students  may  not  have 
the  technical  knowledge  and  skills  that  they  should  have,  five 
"interval  tests"  were  prepared.  Because  of  time  constraints, 
only  the  first  three  of  these  tests  could  be  administered  to 
students.  The  first  interval  test  was  administered  to  106 
students  after  they  had  completed  Annex  B,  Radiotelephone 
Procedures.  The  second  interval  test  was  administered  to  47 
students  who  had  completed  Annex  H  and  the  next  annex  in  the 
training  sequence,  Annex  J,  Teletypewriter  Communications 
Procedures.  The  third  interval  test  was  administered  to  11 
students  who  had  completed  Annexes  H,  J,  and  the  third  annex 
in  the  training  sequence,  Annex  X,  Radio  Set  AN/GRC-106.  It 
should  be  noted  that  all  three  tests  focused  on  materials  taught 
in  the  course  and  which  one  might  expect  the  students  to  know 
as  a  result  of  the  training  they  had  received. 


Tables  6,  7,  and  8  on  the  following  pages  show  an 
abbreviated  fora  of  the  test  item,  the  test  item  format 
(FX  ■  fill-in,  M  ■  matching,  MC  ■  multiple  choice) ,  and  the 
percent  of  students  answering  each  test  item  correctly. 

It  is  reasonable  to  assume  that  no  more  than  201  of  the 
students  should  Incorrectly  answer  questions  on  material  that 
they  have  just  been  taught.  As  can  be  seen  in  the  three 
tables,  the  error  rate  far  axceeds  the  modest  20%  standard. 

On  Interval  Test  #1,  the  failure  rate  exceeded  20%  on  34  of 
the  40  scorable  items.  The  corresponding  values  for  Interval' 
Test  #2, are  13  out  of  33.  On  Interval  Test  #3,  12  out  of  21 
items  exceeded  the  20%  failure  rate. 

The  data  in  the  three  tables  should  be  useful  to  members  of 
the  05C  faculty  in  identifying  weaknesses  in  the  current 
training  program.  For  example,  the  table  for  Interval  Test  #1 
reveals  that  students  have  far  more  trouble  in  finding  call 
signs  and  suffixes  in  a  CEOI  than  they  have  in  finding 
frequencies .  This  same  table  also  shows  that  students  are 
able  to  match  selected  prowords  with  their  meanings  and  are 
able  to  use  an  authentication  table.  On  the  other  hand,  the 
table  also  shows  that  between  two-thirds  and  three-fourths  of 
the  students  did  not  know  how  to  respond  correctly  to  requests 
for  repetition. 
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It  should  be  remembered  that  the  interval  tests  are  pencil 
and  paper  tests  that,  in  most  instances,  only  roughly 
approximate  how  the  knowledge  or  skill  is  actually  used  on 
the  job.  While  it  is  possible  that  some  students  will  be 
able  to  perform  the  task  though  they  cannot  answer  questions 
about  the  task,  it  is  more  likely  that  students  who  cannot 
answer  questions  about  the  task  also  will  not  be  able  to 
perform  the  task. 
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1. 

Use 

CEOI 

a. 

Find  NCS  call  sign  and 
suffix 

FI 

60 

b. 

Find  NCS  frequency 

FI 

23 

2. 

US* 

CEOI 

a. 

Find  Commander's  call 
sign  and  suffix 

FI 

52 

b. 

Find  frequency 

FI 

26 

3. 

Use 

CEOI  and  DA  Form  4004 

a. 

Find  and  enter  call  sign 
of  originator  on  form 

FI 

55 

b. 

Find  and  enter  call  sign 
of  action  addressee  on 
form 

FI 

55 

4. 

Match  proword  with  meaning 

a. 

SAY  AGAIN 

M 

22 

b. 

INTERROGATIVE 

M 

18 

c. 

ALL  AFTER 

M 

9 

d. 

WORD  BEFORE 

M 

6 

e. 

THIS  IS 

M 

3 

5. 

Operator  questions  "time"  in 
text  of  message 

MC 

42 

6. 

Operator  requests  repetition 
of  missing  information  in  text 

of  message 

MC 

74 

7. 

NCS 

transmissions 

a. 

To  open  a  net 

FI 

24 

b. 

To  close  a  net 

FI 

23 

2 


Authenticate  a  challenge  using 
an  Authentication  Table 

a.  #1 

b.  #2 

c.  *3 

Enter  Operator's  Service  on 
DA  Form  4004 

Definition  of  Message  Number 

Sequence  of  elements  in  a 
data- time-group 

Elements  in  an  Operator's 
Service. 

Detect  problems  in  messages 
and  requests  repetition 

a.  Find  error  in  text  of  message 

b.  Request  repetition  of 
incorrect  word 

c.  Find  omission  in  text 
of  message 

d.  Request  repetition  of 
omitted  word 

Respond  to  request  for 
repetition. 

a.  Repeats  word  sent/received 
incorrectly 

b.  Repeats  word  omitted 

Operator  questions  "time”  in 
text  of  message 

Fill-out  a  MIJI  Form 

a.  Victim's  call  sign 

b.  Victim's  function 

c.  Type  of  incident 

d.  Operator's  name  and 
function 

e.  Nomenclature  of  equipment 
effected 

f.  Frequency  or  channel  affected 
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TABLE  6  (cont'd) 


TEST  PERCENT  OF 

TEST  ITEM  STUDENTS 

_ ITEM _ FORMAT  INCORRECT 

17.  Fill-out  DA  Form  4004  with  aid 
of  a  CEOI 


a.  Originator's  call  sign/suffix 
changed  to  plain  language 

FI/M 

58 

b.  Action  addressee 'a  call  sign/ 
suffix  changed  to  plain  language 

FI/M 

57 

c.  Signature  of  drafter 

FI/M 

26 

d.  Grade  of  drafter 

FI/M 

25 

e.  Date- time-group 

FI/M 

16 

18. 

Make  entries  on  Operator's 

Number  Sheet 

a.  For  a  message  received 

FI 

34 

b.  For  a  message  sent 

FI 

26 

19. 

Procedure  to  correct  an  error 
on  Operator's  Log  or  Number 

Sheet 

MC 

8 

20. 

The  time  when  the  operator's 
signature  appears  on  the 

Operator's  Number  Sheet 

MC 

12 

21. 

The  time  when  a  new  Operator's 

Log  is  started? 

MC 

37 

22. 

The  purpose  of  the  Circuit  Log 

MC 

78 

45 


TABLE  7  (cont'd.) 


TEST 

ITEM 


TEST  PERCENT  OF 
ITEM  STUDENTS 
FORMAT  INCORRECT 


TABLE  8 

INTERVAL  TEST  #3: 

TEST  ITEM,  TEST  ITEM  FORMAT,  AND  PERCENT  OF  STUDENTS 
ANSWERING  TEST  ITEM  INCORRECTLY 


TEST  PERCENT  OF 

TEST  ITEM  STUDENTS 

ITEM _ FORMAT  INCORRECT 


1.  Action  taken  by  operator  in  RWI 
system  after  conversation  is 
completed. 

2.  Number  of  batteries  used  in 
SB-22/PT  switchboard 

3.  Identification  number  of 
batteries  used  in  SB-22/PT 
switchboard. 

4.  Locate  telephone  switchboard 
designators  and  telephone 
numbers  in  CEOI  extract. 


MC 

18 

MC 

72 

MC 

18 

a.  #1 

FI 

45 

b.  #2 

FI 

54 

c.  #3 

FI 

45 

d.  *4 

FI 

45 

5. 

Fill-out  DA  Form  2404  using 

Table  from  a  TM. 

a.  Nomenclature  and  model 

FI/M 

0 

b.  Serial  number 

FI/M 

0 

c.  Date 

FI/M 

9 

d.  Type  of  inspection 

FI/M 

0 

e.  TM  number 

FI/M 

45 

f .  TM  Date 

FI/M 

0 

g.  TM  Item  number 

FI/M 

54 

h.  Deficiencies  and 
shortcomings 

FI/M 

0 

6. 

Setting  of  VOX  switch  on 
AN/GRC-106  radio  to  transmit 

MC 

0 

7. 

Function  of  frequency  vernier 
on  an  AN/GRC-106  radio 

MC 

27 

8. 

Control  on  AN/GRC-106  radio  to 
use  to  reduce  noise  level. 

MC 

36 

TABLE  8  (cont'd.) 


TEST 

ITEM 

TEST 

ITEM 

FORMAT 

PERCENT  OF 

STUDENTS 

INCORRECT 

9. 

Connection  at  one  end  of  the 

RF  cable  assembly  of  the 
AN/GRC-106  radio  when  used 
in  the  receive  mode. 

MC 

36 

10. 

Connection  at  one  end  of  the 

RF  cable  assembly  of  the 
AN/GRC-106  radio  when  used  in 
the  transmit  mode. 

MC 

27 

11.  Effect  of  having  bare  ends 
of  wire  connecting  line 
terminals  of  the  AN/GSA-7 

make  contact  with  each  other  mc  63 


9.  RESULTS  OF  THE  VERIFICATION  OF  BASIC  SKILLS 


One  approach  to  verifying  the  basic  skills  selected  as  pre¬ 
requisites  for  inclusion  in  FBSEP  is  to  correlate  scores  on 
the  Diagnostic  Test  with  success  in  the  05C  AIT  Course.  How¬ 
ever,  caution  should  be  exercised  regarding  this  approach  be¬ 
cause  the  Diagnostic  Test  is  not  a  perfect  instrument  for 
measuring  the  basic  skills  identified.  Due  to  the  constraints 
of  time  allowed  for  administering  the  test  (2  hours),  the  test 
scores  should  be  considered  a  sample  measurement  of  the  basic 
skills. 

Table  shows  the  correlations  between  the  Diagnostic  Test 
and  its  subsections  with  student  status  in  the  course  at  the 
end  of  Annex  J,  the  annex  concerned  with  sending  messages  on 
the  teletypewriter.  Student  status  was  dichotomized  as  success 
or  failure.  Students  who  were  dropped  from  the  course  for 
academic  or  other  reasons,  at  the  end  of  Annex  J  or  earlier, 
were  classified  as  failures.  Students  who  passed  the  Annex  J 
test,  and  all  earlier  tests,  were  classified  as  successes. 

It  should  be  noted  that  while  some  students  are  dropped  from 
the  course  prior  to  Annex  J,  and  some  in  Annex  J,  few  if  any 
drops  occur  later  in  the  course. 


TABLE  9 


CORRELATION  OF  DIAGNOSTIC  TEST  SCORES  WITH  SUCCESS 

IN  THE  05C  COURSE 


CORRELATION 

Diagnostic  Test 

.12* 

Hath  subsection 

.11* 

Language  subsection 

.05 

Reading 

.09 

N-325 

*p  < .  05 

While  these  correlations  are  low,  those  for  the  test  as  a  whole 
and  for  the  math  subsection  are  significant.  If  we  had  assumed 
an  underlying  continuous  distribution  for  success  in  the  05C 
course  instead  of  a  dichotomous  distribution,  we  could  have 
computed  biserial  correlations  which  are  usually  higher  than 
the  point  biserial  correlation  used  here.  In  other  words,  it 
is  postulated  that  the  true  correlations  are  higher  than  those 
presented  here,  and  the  correlation  for  reading  would  be 
significant  ( . 13 ) . 

It  also  should  be  noted  that  the  "failure”  category  includes 
students  who  were  dropped  from  the  course  for  administrative 
reasons .  If  the  data  were  limited  to  success  and  failure  on 
academic  performance  only,  the  correlations  might  well  be 
higher . 


Once  the  05C  FBSEP  is  implemented  data  should  be  collected  on 
the  students  who  90  through  their  assigned  basic  skills  lessons 
to  see  if  the  failure  rate  in  the  05C  course  is  reduced.  This 
will  be  the  final  verification  of  the  basic  skills. 

Procedures  for  Eliminating  Invalid  Data 

The  following  precautions  were  taken  to  ensure  the  accuracy  of 
the  data  collected. 

1.  Predictor  measures 

a.  Diagnostic  Test  total  scores  and  subsection  scores 
were  checked  visually  for  values  outside  the  possible 
score  range.  Since  this  data  was  collected  on  Form 
C  of  the  test,  the  possible  ranges  for  the  subtests 


and 

total  tests 

are  as  follows 

#  right 

1. 

Language 

0-22 

2. 

Math 

0 

1 

M 

00 

3. 

Reading 

0-70 

4. 

Total  Test 

0-120 

b.  Student  Questionnaire  items  were  checked  to  see  that 
all  items  were  answered  and  that  only  one  choice  was 
selected.  This  check  was  made  by  the  proctors  when 
the  student  handed  in  his  Diagnostic  Test  answer 
sheet.  If  there  was  a  problem  it  could  be  handled 
immediately. 


c.  ASVAB  scores  ware  checked  to  sea  that  all  raquired 
subtast  scoras  are  available.  Since  no  student 
scoring  below  95  in  the  SC  subtest  may  take  the 
05C  course,  all  SC  scores  were  checked  to  see  that 
they  were  above  95.  Also  a  maximum  score  of  135  is 
possible.  Therefore,  these  scores  must  be  within 
a  range  of  95-135. 

Criterion  measures  -  Information  obtained  from  the  05C 
course  summary  sheets  were  checked  against  the  data 
collected  from  the  instructors.  For  example,  if  the 
summary  sheet  indicated  that  a  student  had  two  retests, 
both  retests  should  be  available  in  the  form  of  check¬ 
lists  for  end-of-lesson  tests.  These  test  results  were 
compared  to  the  printouts  obtained  from  ATOMS  to  see 
if  there  were  differences.  If  differences  occurred 
they  were  rectified  before  being  key-punched.  . 


10.  ANALYSIS  OF  BASELINE  DATA  AND  FAILURES 


Date  was  collected  on  270  students  who  failed  to  complete  the 
OSC  AIT  course  prior  to  the  onset  of  FBSEP.  This  section  of 
the  Interim  Report  describes  some  of  the  characteristics  of 
these  students. 

Academic  vs  Administrative  Failures 
On  the  basis  of  information  available  to  the  FBSEP  team  located 
at  Ft.  Gordon,  students  who  failed  to  complete  the  05C  AIT 
course  were  classified  as  either  an  academic  failure  or  an  ad¬ 
ministrative  failure.  A  student  was  classified  as  an  academic 
failure  if  it  was  judged  that  the  reason  for  failure  was  lack 
of  aptitude  or  lack  of  prerequisites.  A  student  was  classified 
as  an  administrative  failure  if  it  was  judged  that  the  reason 
for  the  failure  was  lack  of  motivation  or  personal  problems  which 
interfered  with  academic  performance.  While  it  is  recognized 
that  often  one  affects  the  other  and  that  clear  distinctions 
cannot  always  be  made  between  the  two,  the  data  we  have  may  prove 
useful  to  course  administrators  and  faculty.  The  date  is  pre¬ 
sented  in  Table  10. 


N 

% 

Academic  Drop 

192 

71 

Administrative  Drop 

54 

20 

Unknown 

24 

9 

270 

1004 

It  is  apparent  that  the  major  reason  for  being  dropped  from  the 
course  was  judged  to  be  academic/  the  student  lacking  the  aptitude 
or  the  prerequisites  needed  to  succeed  in  the  course. 

Annex  Difficulty 

Using  available  data,  the  FBSEP  team  at  Ft.  Gordon  also  made  a 
judgment  regarding  the  particular  05C  AIT  annex  in  which  the 
students  who  were  dropped  from  the  course,  regardless  of  reason, 
experienced  the  siost  difficulty.  Two  categories  of  "difficulty" 
were  established,  primary  and  secondary.  Table  11  shows  which 
annexes  were  judged  to  be  of  primary  and  secondary  difficulty 
for  the  student. 


0(900  HHH  *i  ►}  *->  OBb 


Primary 

Difficult’ 


Secondary 

Difficult' 


H  t  J 
H 
J 

Sub  total 


G 

J 

Sub  total 


G  &  H 
G 
J 

Sub  total 
Grand  total 


Primary  or  secondary 
Primary  or  secondary 
Primary  or  secondary 
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Zt  is  evident  from  the  table  that  the  annex  judged  to  cause  the 
most  difficulty  was  Annex  G,  typing.  Annexes  H  (Radio  Voice 
Procedures)  and  J  (Radio  Teletypewriter  Procedures)  were  judged 
to  be  about  equally  difficult  for  students.  It  should  be  noted 
that  typing  also  is  an  important  component  of  Annex  J. 

Attitudes  of  Students 

Students  who  were  dropped  from  the  course  were  given  15  cards, 
each  containing  a  different  problem  which  could  affect  their 
performance  in  the  course.  The  student  were  directed  to  select 
those  cards  (problems)  which  best  explained  their  failure  in 
the  course.  The  results  are  shown  in  Table  12. 

It  is  apparent  that  the  problem  most  frequently  reported  by 
students  was, "I  could  not  learn  to  type.”  This  finding  is  con¬ 
sistent  with  the  judgment  of  FBSEP  personnel  at  Fort  Gordon 
that  the  annex  which  caused  primary  difficulty  for  students  was 
Annex  G,  Typing. 


_ Problem _ N _ %_ 

I  could  not  learn  to  type .  163  25 

I  could  not  learn  the  material . 76  12 

I  want  to  be  reclassified  to  another  MOS . 71 

I  had  personal  problems  at  home . 59  9 

The  recruiter  lied/mislead  about  the  MOS . 53  8 

I  dislike  the  army . 40  6 

I  could  not  understand  the  instructor . 34  5 

I  do  not  like  a  self  paced  course . 28  4 

I  had  too  much  lost  time  from  course . 25  4 

Company  life:  Mo  sleep,  barracks  life,  duty, 

extra  duty,  etc . 22  3 

I  do  not  like  to  be  in  a  classroom . 22  3 

There  were  too  many  distractions  in  self  paced  class.  21  3 

I  have  physical  defects /health  problems . 21  3 

I  could  not  get  along  with  the  instructor . 19  3 

I  have  religious  convictions  which  will  not  allow  me 

to  participate  in  the  army .  6  1 


100 


TOTAL 


660 


The  course  method  of  calculating  attrition  rates  has  been  to 
divide  the  percentage  of  students  leaving  the  course  each  week 
without  graduating  by  the  total  number  of  graduates  plus  attriter 
for  that  week: 


attrites 


graduates  +  attrites 


percentage 


This  formula  has  the  weakness  that  it  does  not  have  a  relation¬ 
ship  to  the  course  load  or  inputs. 


We  have  tracked  the  attrition  rate  thoughout  our  entire  contract 
period  and  have  calculated  an  attrition  rate  for  each  week. 

This  attrition  rate  is  calculated  by  first  matching  each  student 
with  their  input  group  and  then  calculating  a  percentage  of  each 
group  that  has  failed  the  course.  This  process  allows  us  to  see 
how  many  students  from  each  group  fail  the  course. 


Data  on  size  of  class  and  number  of  students  dropped  from  each 
class  was  collected  for  46  classes,  starting  on  10  July  1981 
and  ending  on  11  June  1982.  Figure  5  presents  the  data  in  graphic 
form.  The  upper  line  shows  the  number  of  students  that  started 
training  on  the  given  date.  Starting  class  size  ranged  from 
twelve  on  24  July  1981  to  119  on  18  September  1981.  The  lower 
line  shows  the  number  of  students  that  were  dropped  from  that 
class,  that  is,  did  not  graduate.  The  number  of  "drops”  ranged 


n » 


from  0  for  the  class  which  started  training  on  17  July  1981  to 
20  for  tha  class  which  started  training  on  5  February  1982. 

For  purposes  of  analysis,  the  46  classes  were  divided  into  three 
groups  on  the  basis  of  their  enrollment  at  the  start  of  training: 
12-45  students,  46-60  students,  and  61-119  students.  The  number 
of  students  dropped  from  the  classes  in  each  groups  was  then 
determined.  (Appendix  A  relates  the  attrition  rate  to  the  per¬ 
centage  of  students  dropped  for  each  week.)  The  data  is  pre¬ 
sented  in  Table  13. 
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TABLE  13 

MEAN  CLASS  SIZE,  MEAN  NUMBER  OF  STUDENTS  DROPPED,  AND 


PERCENT  DROPPED  FOR  46 

05C  AIT 

CLASSES 

Class 

Size 

12-45 

46-60 

61-119 

Number  of  classes 

15 

16 

15 

Mean  size  of  each  class 

32.7 

52.2 

78.9 

Mean  number  of  students  dropped 

5.5 

4.4 

7.9 

Percent  of  students  dropped 

17% 

8% 

10% 

The  table  shows  that  while  there  were  more  students  dropped  from 
the  largest  classes  (61-119  students)  than  from  the  smaller 
classes  (12-45  and  46-60  students),  the  drop  rate  for  the  largest 
classes  was  about  the  same  as  for  middle-size  classes  and  lower 
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than  for  the  smallest  classes.  It  thus  appears  that  increasing 
the  size  of  the  class  does  not  result  in  an  increase  in  the  drop 
rate.  However,  in  terms  of  number  of  students  failing  out  of 
the  course,  the  number  is  smaller  when  the  class  size  is  under 


11.  DIFFICULTIES  ENCOUNTERED  AND  PRINCIPAL  FINDINGS 


This  section  of  the  report  presents  the  difficulties  encountered 
in  the  research  and  development  effort  which  culminated  in  the 
Analysis  Report  and  dilineates  the  principal  findings  of  this 
research  and  development  effort. 

Difficulties  Encountered 

This  section  describes  the  difficulties  encountered  in  conduct¬ 
ing  the  research  and  development  effort.  Certain  difficulties 
were  more  evident  for  the  San  Diego  team  than  for  the  Ft.  Gordon 
team,  because  the  San  Diego  team  did  not  have  ready  access  to 
the  equipment  and  subject  matter  experts  located  at  the  Signal 
School.  The  problems  were  generally  solved  by  contacting  the 
Signal  School  team  and  asking  them  to  examine  the  equipment  and/ 
or  consult  subject  matter  experts  on  the  faculty. 

1.  In  the  course  of  examining  AIT  training  materials  in  order 
to  identify  gaps  and  prerequisite  competencies ,  the  analyst 
should  have  ready  access  to  the  equipment  (radios,  tele¬ 
typewriter,  antennas,  etc.)  which  is  the  subject  matter  of 
the  AIT  lessons.  Having  this  equipment  at  hand  would  allow 
the  analyst  to  more  quickly  recognize  gaps  in  the  training 
materials  and  identify  the  basic  skills  which  are  pre¬ 
requisite  to  the  AIT  lesson. 
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Related  to  this  was  the  unavailability  of  certain  Learning 
Supervisor  Guides  and  Technical  Manuals,  and  restricted 
access  to  classified  training  areas.  These  limitations 
complicated  the  task  of  developing  present  and  recommended 
heirarchical  structures  for  each  lesson. 

The  lack  of  consistency  between  different  training  resources 
(lesson  study  guides,  training  circulars,  CEOI,  and  TEC) 
also  severely  compounded  the  difficulties  encountered  in 
developing  both  the  present  and  recommended  heirarchical 
structures  for  each  lesson. 

Still  other  factors  compounding  the  analysis  procedure  were 
ambiguities  and  gaps  in  available  training  materials  and 
the  fact  that  some  of  the  materials  made  available  to  the 
analysts  was  obsolete  and  had  already  been  superseded  by 
more  current  or  revised  materials.  As  noted  earlier,  these 
problems  were  overcome,  but  their  presence  hampered  the 
analysis  process. 

When  analyzing  training  materials  and  preparing  present 
and  recommend  heirarchical  structures  for  each  AIT  lesson, 
analysts  attempted  to  classify  the  behavior  that  the  student 
was  expected  to  learn  into  the  following  nine  categories: 


Know  fact 

Know  concapt 
Use  concept 

Know  procedure 
Use  procedure 

Know  rule 
Use  rule 

Know  principle 
Use  principle 

Despite  definitions  and  examples  of  each  category,  it 
frequently  proved  difficult  to  make  definitive  categoriza¬ 
tions.  Particular  difficulty  was  encountered  in  distinguish¬ 
ing  between  "know  fact,"  "know  concept,"  and  "know  rule." 

It  should  be  remembered  that  these  categories  are  not  present 
in  the  existing  study  material  and  that  it  was  the  analyst 
who  tried  to  superimpose  them  on  the  existing  structure  and 
incorporate  them  to  the  recommended  structure.  While 
categorizations  were  made,  we  have  reservations  about  the 
validity  and  reliability  of  the  results.  We  doubt  that 
these  weaknesses  will  have  any  affect  on  the  final  outcome 
of  the  research  and  development  effort. 

6.  Another  problem  encountered  in  developing  heirarchical 
structures  for  the  AIT  lessons  was  noting  behaviors  that 
had  been  presented  in  earlier  lessons.  As  has  been  described 
earlier,  such  behaviors  were  annotated  as  being  "remember" 
items.  This  constituted  a  problem  because  turnover  among 
analysts  and  the  assignment  of  different  analysts  to 
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different  lessons  resulted  in  lack  of  continuity.  If  the 
analyst  had  not  examined  the  earlier  lesson  in  which  the 
behavior  had  first  been  taught,  he  had  no  reason  to  cite 
its  subsequent  appearance  in  a  later  lesson  as  a  "remember" 
item. 

In  analyzing  training  materials  in  order  to  develop  heir- 
archical  structures,  and  identify  gaps  in  technical  content 
and  prerequisite  competencies,  the  analysts  examined  the 
Soldier's  Manual,  the  Lesson  Study  Guides,  the  Learning 
Supervisor's  Guides,  and  the  mandatory  elements  in  the  end 
of  course  test.  Ideally,  there  should  be  a  consistency 
between  the  content  in  all  these  documents.  Unfortunately 
this  was  not  always  true.  Because  of  content  discrepancies 
in  the  documents,  the  analyst  relied  most  heavily  on  the 
Lesson  Study  Guides  and  Learning  Supervisor's  Guides  in 
making  their  analysis.  This  was  done  because  these  two 
documents  most  completely  describe  what  the  Army  has  decided 
the  student  should  be  taught  in  the  05C  AIT  Course,  the 
fccus  of  our  attention  in  the  FBSEP . 

Both  the  Lesson  Study  Guide  and  Learning  Supervisor's  Guide 
for  each  lesson  list  the  identification  number  of  Soldier's 
Manual  tasks  which  are  related  to  the  tasks  covered  in  the 
lesson.  While  there  should  be  close  agreement  in  the 
Soldier's  Manual  tasks  so  cited,  this  is  not  always  the  case. 


As  can  b«  seen  in  Table  14,  in  only  31%  of  the  entries  is  there 
complete  agreement  between  the  Soldier's  Manual  tasks  cited  in 
Lesson  Guides  and  Soldier's  Manuals  tasks  cited  in  the  Learning 
Supervisor's  Guides  for  the  same  lesson.  Detailed  information 
is  found  in  Appendix  B. 

TABLE  14 

FREQUENCY  AND  PERCENT  OF  SOLDIER'S  MANUAL  TASKS  CITED  IN 
LESSON  STUDY  GUIDES  AND  LEARNING  SUPERVISOR'S  GUIDES 


N 

% 

Same  task  cited  in  both  Lesson  Study 
Guide  and  Learning  Supervisor's  Guide. 

Task  cited  in  Lesson  Study  Guide  only. 

37 

31 

44 

36 

Task  cited  Learning  Supervisor's 

31 

26 

Guide  only. 

Task  cited  in  Lesson  Study  Guide. 

8 

7 

No  Learning  Supervisor's  Guide 
available. 

TOTAL 

120 

100 

Table  15  shows  that  a  number  of  Soldier's  Manual  Tasks  are  not 


cited  in  either  a  Lesson  Study  Guide  nor  a  Learning  Supervisor's 
Guide  as  being  "related"  to  an  05C  AIT  course  lesson.  Two  of 
the  tasks  listed  actually  are  taught  in  the  05C  AIT  Course,  Pre¬ 
pare  An  Interference  Report  is  taught  in  lesson  H-06  and  Install 
and  Operate  Communication  Equipment  TSEC/KY-38  is  taught  in 
lesson  N-03.  Presumably  the  other  tasks  are  to  be  learned  on- 
the-job. 


SKILL  LEVEL  1 


TASK  NUMBER  TASK  TITLE 

113-573-4001  Encode  and  Decode  Messages  Using  KTC  600D 

Tactical  Operations  Code 

113-573-7001  Prepare  an  Interference  (MIJI)  Report 

113-573-8005  Use  an  Automated  CEOI 

113-587-2002  Operate  Radio  Set  AN/VRC-64 

113-587-2004  Troubleshoot  AN/VRC-46  (AN/VRC-12  Series) 

113-587-3001  Perform  Operator's  Preventive  Maintenance 

on  Radio  Set  AN/PRC-77  or  AN/PRC-25 

113-599-3001  (RC)  Perform  Weekly  Preventive  Maintenance 

on  Radio  Teletypewriter  Set  AN/GRC-46 

113-609-1003  Install  and  Operate  Communications  Equipment 

TSEC/KY-38 

113-609-2013  Operate  TSEC/KY-57  in  Cipher  Test  Mode 
113-620-1003  Install  Radio  Set  AN/FRC-93 

113-620-2005  Operate  Radio  Set  AN/FRC-93 

113-620-1004  (SF)  Install  Radio  Set  AN/PRC-74B 

113-620-2006  (SF)  Operate  Radio  Set  AN/PRC-74B 

113-622-2001  Operate  Radio  Set  Control  Group  AN/GRA-6 
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Task  Number 


113-571-1005 

113-573-8001 

113-573-6002 

113-623-3002 

120-030-1503 


Number  of  Lessons  In  Which  Listed 


Table  17  shows  which  lessons  in  the  course  POI  and  which 
tasks  in  the  Soldier's  Manual  are  in  direct  support  of 
each  of  the  10  "mandatory  elements"  on  the  end-of -course- 
test.  To  some  extent,  the  "mandatory  elements”  rather 
than  the  Lesson  Study  Guides  or  Learning  Supervisor's 
Guides  determines  the  emphasis,  if  not  the  content,  of 
instruction  in  the  course. 


It  should  be  noted  that  the  two  lessons  in  Annex  K  support 
an  "optional”  but  no  "mandatory  elements"  on  the  end  of 


course  test. 


1.  Touch  type  on  teletypewriter  keyboard  at  25  net  WPM  in 
message  format. 

Course  POI  G-01  Keyboard  Techniques 

Soldier's  Manual 


4.  Maintain  Station  Log. 

Course  POI  H-04  Maintaining  Circuit  Log  and 

Operator's  Number  Sheet 

Soldier's  Manual 


5.  Employ  radio  teletypewriter  procedures. 

Course  POI  J-01  Establishing  Radio  Teletypewriter 

Communications 

J-02  Preparing  Teletypewriter  Messages 
J-03  Transmitting  Requests  and  Responses 

Soldier *8  Manual  113-573-1002  Prepare  a  Message  For 

Transmission  in  16- 
Line  Format 


7.  Employ  radio  telephone  procedures. 

Opening  And  Closing  A  Radio  Tele¬ 
phone  Net  Using  Radio  Set  AN/PRC-77 
Preparing  Radio  Telephone 
Messages  for  Transmission 
Transmitting  Messages  in  a 
Radio  Telephone  Net 

Soldier's  Manual  113-571-1003  Establish,  Enter  or  Leave 

A  Radio  Net 

113-571-1001  Transmit  or  Receive  A 
Radio  Message 


Course  POI  H-01 

H-02 

H-03 


8.  Employ  electronic  counter  countermeasures  (EC CM) 


Course  POI  H-06  Using  Electronic  Counter-Counter 

Measures  (EC CM)  on  FM  Radio 
Equipment 

Soldier’s  Manual  113-573-6001  Recognize  Electronic 

Countermeasures  (ECM) 
and  Implement  Electronic 
Counter-Countermeasures 
(ECCM) 


10.  Install  speech  security  equipment  TSEC/KY-8 

Course  POI  L-03  Installing  Speech  Security  Equipment 

Soldier's  Manual  113-609-1001  Install  and  Operate 

Communication  Security 
Equipment  TSEC/KY-8 


14.  Install  and  operate  radio  teletypewriter  set  AN/GRC-142-122 

Course  POI  M-02  Install  and  operate  radio  teletype¬ 

writer  set  AN/GRC-142  or  122 

Soldier's  Manual  113-599-1002  Install  Radio  Teletype- 

writer  Set  AN/GRC-142  or  122 
113-599-2002  Operate  Radio  Teletype¬ 
writer  Set  AN/GRC-142  or  122 


18.  Install  and  operate  KWI  system 

Course  POI  1-01  Installing  and  Operating  Radio 

Wire  Integration  System 

Soldier's  Manual  113-618-1001  Install  Radio  Wire 

Integration  (RWI)  System 
113-618-2001  Operate  Radio  Wire 

Integration  (RWI)  System 


TABLE  17  (cont'd.) 


20.  Install  antenna  AN/GRA-50 

Course  POI  N-03  Installing  Antenna  Group  AN/GRA-50 

Soldier's  Manual  *113-596-7001  Inspect  Contruction  Of 

A  Doublet  Antenna 


22.  Install,  operate,  perform  operator's  maintenance  and  trouble¬ 
shoot  generator  10KW  120V  ac. 

Course  POI  N-05  Installing  and  Operating  Generator 

Set  10KW  120V  ac 

N-06  Preventive  Maintenance  and  Trouble¬ 
shooting  Generator  Set  10KW  120V  ac 


Soldier's  Manual 


113-601-1002 

113-601-2003 

113-601-3002 


113-201-2004 


Install  Generator  Set  10KW 
Operate  Generator  Set  10KW 
Perform  Daily  Preventive 
Maintenance  Checks  and 
Services  on  Generator  Set 
10KW 

Perform  Operator's  Trouble¬ 
shooting  Procedures  on 
Generator  Set  10KW 


•Listed  in  the  Soldier's  Manual  as  Skill  Level  2. 


8.  Ideally  the  Soldier's  Manual  specifies  the  tasks  in  which 
a  person  of  a  particular  MOS  and  skill  level  siust  be  pro¬ 
ficient  and  the  AIT  course  prepares  the  student  to  perform 
the  tasks  using  the  training  objective  as  specified  in  the 
Soldier's  Manual  as  the  guide  for  course  instruction.  The 
relation  between  the  Soldier's  Manual  and  course  lessons 
for  MOS  05C  is  not  that  clear.  The  following  example 
illustrates  this  point. 

Soldier's  Manual  Task  113-571-1001  is  titled  Transmit  or 
Receive  A  Radio  Message.  The  following  05C  AIT  course 
lessons  cite  the  above  Soldier's  Manual  task  as  being 
related : 

F-02  Using  Alphanumerics 

H-02  Preparing  Radio  Telephone  Messages  For 
Transmission 

H-03  Transmitting  Messages  in  a  Radiotelephone  Net 

H-04  Maintaining  Circuit  Log  and  Operator's  Number 
Sheet 

H-05  Install  and  Operate  Radio  Set  AN/VRC-12  Series 

1-01  Installing  and  Operating  Radio  Hire  Integration 
System 

J-03  Transmitting  Requests  and  Responses 

K-01  Installing  and  Operating  Radio  Set  AN/GRC-106 
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The  term  "related"  is  not  defined  and  in  most  instances, 
the  relation  between  the  task  in  the  Soldier's  Manual  and 
the  lessons  in  the  course  is  tenuous.  Lessons  F-02  and  H-02 
are  clearly  related  to  the  Soldier's  Manual  task,  both  in 
the  content  of  the  instruction  and  in  the  practice  exercises 
which  follow  the  lesson.  Lessons  H-03  and  J-03  are  concerned 
requests  for  repetition  and  responding  to  requests  for 
repetition,  two  tasks  which  are  hardly  noted  in  the  Soldier's 
Manual  Task  and  which  constitute  very  narrow  and  specialized 
aspects  of  transmitting  and  receiving  messages.  Lessons 
H-05,  1-01  and  K-01  require  the  student  to  transmit  and 
receive  a  message  as  part  of  the  practice  exercise  which 
follows  the  lesson,  but  we  suspect  that  transmitting  and 

• 

receiving  messages  in  these  three  lessons  is  minimal,  and 
incidental  to  installing  and  operating  the  equipment. 

Lesson  H-04  includes  no  requirement  to  transmit  or  receive 

a  message. 

The  discrepency  between  Soldier’s  Manual  tasks  and  05C  AIT 
course  lessons  is  also  evident  when  one  examines  the  com¬ 
ponents  of  both  the  Soldier's  Manual  task  and  the  05C  AIT 
course  lessons  specifically  concerned  with  transmitting 
and  receiving  messages  (F-02  and  H-02). 


An  examination  of  the  performance  measures  listed  under 
the  Soldier's  Manual  task  reveals  that  this  task  encompasses 
five  elements: 

1.  transmitting  prowords 

2.  transmitting  isolated  letters  and  abbreviations  using 
the  phonetic  alphabet 

3.  phonetically  spelling  unusual  or  difficult  words 

4.  transmitting  numerals  phonetically  (digit  by  digit) 

5.  transmitting  numbers  that  are  exact  multiples  of 
thousands  (not  digit  by  digit) 

Following  a  discussion  of  the  five  above  elements ,  the 

Soldier's  Manual  task  presents  a  sample  transmission  in* 

corpora ting  these  elements. 

The  following  discrepancies  occur  between  the  specifications 
in  the  Soldier's  Manual  and  the  specifications  in  course 
lessons  F-02  and  H-02* 

1.  Neither  course  lesson  makes  any  reference  to  the 
transmission  of  numerals  other  them  digit  by  digit. 

2.  The  sample  transmission  in  the  Soldier's  Manual 
omits  many  elements  found  in  the  sample  message 
included  in  05C  AIT  lesson  H-02.  Among  these 
omissions  are: 

a.  First  part  of  preliminary  call  of  receiving  station 

b.  First  part  of  message  heading 

c.  Message  number 


3. 


d.  Priority 

e .  Oa  t a- time-group 

f.  From,  To,  Info,  Exempt  (Location  in  message) 

g.  Break  (Location  in  message) 

h.  First  part  of  receiving  station's 
acknowledgement 

The  sample  transmission  in  the  Soldier's  Manual 
uses  abbreviated  call  signs  to  identify  both  the 
transmitting  and  receiving  stations.  Abbreviated 
call  signs  are  mentioned  only  briefly  and  are  not 
used  at  all  in  the  sample  messages  included  in 
lesson  H-02. 

4.  The  sample  transmission  in  the  Soldier's  Manual 
uses  the  phonetic  alphabet  to  show  abbreviations 
and  uses  words  to  indicate  punctuation  in  the 
message.  Neither  of  these  items  are  mentioned  in 
lessons  F-02  or  H-02 


The  reader  may  well  wonder  why  we  have  presented  so  detailed 
a  discussion  of  discrepancies  between  tasks  in  the  Soldier's 
Manual  and  lessons  in  the  05C  AIT  course.  The  reason  is 
that  these  are  the  documents  that  the  analyst  must  use  to 
prepare  his  heirarchical  analysis,  to  identify  gaps  in 
course  content  and  to  identify  prerequisite  competencies. 

If  the  analyst  expects  to  find  a  well  organized  and  clearly 
defined  set  of  training  objectives,  he  may  be  sadly  dis¬ 
appointed.  He  will  often  find  discrepencies  and  incon¬ 
sistencies  which  he  will  have  to  resolve  before  he  can 
proceed  with  his  main  task. 
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9.  After  examining  the  training  materials  for  a  lesson,  the 


analyst  prepared  two  structures,  a  Present  Structure  which 
showed  the  current  arrangement  (sequence)  of  lesson  com¬ 
ponents  and  a  Recommended  Structure  which  showed  the  proposed 
arrangement  (sequence)  of  lesson  components.  As  noted  earlier, 
the  components  in  the  Recommended  Structure  were  not  neces¬ 
sarily  identical  with  those  in  the  Present  Structure.  This 
was  true  when  the  analyst  felt  that  certain  components 
should  be  added  or  deleted. 

Each  structure  was  presented  in  two  forms,  pictoriallv  in 
the  form  of  a  block  diagram  and  textuallv  in  the  form  of  an 
outline.  Two  problems  develped  with  reference  to  the 
pictorial  and  textual  presentation  of  the  structures. 

First,  while  the  block  diagrams  represent  learning  heir- 
archies,  they  are  not  complete.  Strictly  speaking,  a 
"learning  heirarchy"  depicts  all  the  prerequisite  relation¬ 
ships  of  knowledges,  skills,  and  tasks.  For  example,  the 
name  of  a  task  is  shown  within  a  box.  The  names  of  the 
one  or  more  tasks  which  are  directly  prerequisite  to  the 
first  task  are  shown  in  boxes  below  the  first  task.  In 
turn,  the  names  of  the  one  or  more  tasks  which  are  directly 
prerequisite  to  each  second  level  task  are  shown  in  boxes 
directly  below  the  task  to  which  they  apply.  This  pattern 
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continues  until  the  objective  has  been  broken  down  to  it's 
most  basic  subordinate  objectives.  As  examination  of  the 
block  diagrams  reveals,  they  generally  do  not  show  this 
specificity.  They  show  the  terminal  learning  objective  and 
only  two  or  three  lower  levels  of  enabling  objectives. 

The  second  problem  is  with  the  textual  description  of  the 
lesson's  structure.  It  must  be  understood  that  this  too 
is  presented  in  hierarchial  form  with  the  major  objective 
at  the  top  or  stated  first  and  the  lower  level  enabling 
objectives  following.  The  presentation,  therefore,  is  in 
the  reverse  order  of  the  instructional  and  learning  pro- 
cesses  and  should  be  applied  in  a  "botton-up"  sequence 
within  each  section. 

A  major  focus  of  the  analyst  was  to  identify  gaps  in  the 
content  of  the  lesson  material  that  would  serve  as  a  barrier 
to  learning.  Gaps  were  identified  by  carefully  examining 
the  sequence  of  content  in  each  lesson. 

Three  types  of  gaps  were  identified  in  the  lesson  material. 
First,  there  were  gaps  in  the  explanation  of  key  terms  and 
concepts;  that  is,  a  term  or  concept  is  used  but,  in  the 
judgment  of  the  analyst,  inadequately  defined,  explained. 
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or  described  in  the  training  material.  Often,  the  terms 
referred  to  components  of  the  equipment  that  the  student 
was  being  taught  to  install  or  operate. 

A  second  type  of  gap  took  the  form  of  an  ambiguity  in  the 
steps  to  be  followed  in  a  procedure.  The  analyst  read  the 
steps  in  the  Lesson  Study  Guide  and  tried  to  mentally  per¬ 
form  each  step  in  the  procedure.  When  he  encountered  a 
point  in  the  text  where  the  explanation  was  confusing  or 
not  as  clear  as  it  could  be,  it  was  noted.  This  too  was  a 
matter  of  judgment  on  the  anlyst's  part.  In  some  cases, 
the  analyst  was  able  to  clarify  the  ambiguity  from  the  con¬ 
text  of  the  material.  In  other  cases,  it  was  necessary  to 
examine  technical  manuals  or  consult  instructors  to  clarify 
the  material.  Generally,  these  gaps  were  filled  by  inserting 
a  word  or  two,  replacing  a  pronoun  with  a  noun,  repeating  a 
phrase,  clause  or  sentence,  or  rearranging  parts  of  a  sentence. 

The  third  type  of  gap,  occuring  far  less  often  than  the  other 
two,  was  the  omission  of  a  fairly  complex  procedure  needed 
in  order  to  attain  the  lesson's  terminal  objective.  Three 
gaps  of  this  type  were  noted  in  the  training  material.  The 
first  concerned  the  CEOI.  Though  on  the  job  the  soxctier 
will  be  required  to  use  the  CEOI  to  determine  call  sign, 


78 


suffixes,  and  frequencies,  he  is  not  required  to  do  this  in 
the  05C  AIT  course.  When  the  task  requires  the  student  to 
use  a  call  siqn,  suffix,  or  frequency,  he  is  given  this  in¬ 
formation  in  the  form  of  an  "extract  from  a  CEOI."  We  have 
decided  that  it  is  doing  a  disservice  to  the  student  and 
to  his  future  unit  to  leave  this  gap  unfilled.  Accordingly, 
we  have  inserted  instruction  on  finding  information  in  the 
CEOI  in  the  recommended  structure  for  lesson  H-01  and  in¬ 
cluded  instruction  on  finding  information  in  tables  as  one 
of  the  basic  skills  for  the  FBSEP. 

The  second  complex  procedure  omitted  from  the  training 
material  concerns  the  identification  of  information  that  is 
missing  or  in  error.  In  the  05C  course  the  student  is 
taught  how  to  request  repetition  of  missing  or  erroneous 
information  and  how  to  respond  to  requests  for  repetition. 
When  the  student  performs  this  task,  he  is  told  what 
information  is  missing  or  erroneous.  He  is  given  no 
instruction  or  practice  on  deciding  for  himself  whether 
or  not  information  is  missing  or  erroneous.  We  felt  that 
this  gap  too  should  be  filled.  Accordingly  we  have  included 
instruction  on  deciding  if  information  is  missing  or  erro¬ 
neous  among  the  basic  skills  to  be  covered  in  the  FBSEP. 

The  third  complex  procedure  omitted  from  the  training  material 
concerns  erecting  the  doublet  antenna  when  not  using  the 


tripod  adapter.  The  student  is  taught  how  to  prepare  the 
antenna  for  erection  but  not  how  to  actually  erect  it. 
Since,  on  the  job,  the  student  would  be  expected  to  erect 
the  antenna,  we  have  recommended  that  instruction  on  erect¬ 
ing  the  antenna  be  included  in  lesson  N-02.  We  also  have 
given  suggestions  on  how  this  might  be  done  dispite  limited 


resources . 


Principal  Findings 


This  section  summarizes  the  principal  findings  of  the  research 
and  development  effort  up  through  the  preparation  of  the  Analysis 
Report.  The  findings  are  of  two  types,  those  concerned  with 
the  research  and  development  procedure  and  those  concerned  with 
the  data  that  is  the  product  of  that  procedure.  The  following 
are  the  principal  procedural  findings. 

1.  Despite  the  procedural  problems  cited  earlier,  training 
analysts,  using  recognized  analytic  procedures,  can  examine 
training  resources  and,  although  not  expert  in  the  technical 
aspects  of  the  content,  develop  heirarchical  structures  for 
technical  lessons. 

2.  Training  analysts  can  identify  gaps  in  the  technical  content 
of  each  lesson,  identify  basic  skills  which  constitute  pre¬ 
requisite  competencies  for  each  lesson,  identify  poor  sequenc¬ 
ing  of  lesson  content,  and  identify  technical  content  which 
should  be  deferred  or  deleted  from  each  lesson. 

The  data  findings  are  shown  below. 

3.  The  frequency  with  which  the  four  types  of  problems  cited 
in  #2  above  occurred  in  the  25  05C  AIT  lessons  is  shown  in 
the  following  table. 


In  the  judgment  o£  the  anlysts,  there  were  gaps  in  technical 
content  and  a  requirement  for  training  in  basic  skills  in 
every  lesson.  The  specific  types  of  missing  technical  con¬ 
tent  and  their  frequency  of  occurrence  are  shown  below. 


TABLE  19 

FREQUENCY  OF  MISSING  TECHNICAL  CONTENT 


IN  05C  AIT  LESSONS 

r  ' 

rype  Of  Technical  Content 

soncepts 

157 

- 

facts 

98 

.  / 

steps  in  a  procedure 

50 

rules 

20 

ri 

principles 

1 

■ . 

a 

..  ‘ 

r .  . 

r  t 

j 

82 

• 

It  should  be  noted  that  the  number  157  does  not  represent 
157  unique  concepts  because  at  times,  the  same  concept  was 
reported  missing  in  more  than  one  lesson.  The  other 
frequencies  represent  unique  entries. 

The  specific  types  of  required  basic  skills  and  their 
frequency  are  shown  in  the  following  table. 

TABLE  20 


FREQUENCY  OF  REQUIRED  BASIC 

IN  05C  AIT  LESSONS 

SKILLS 

Type  Of  Basic  Skill 

Concepts 

119 

Procedures 

91 

The  frequencies  shown  do  not  represent  unique  entries  be¬ 
cause  the  same  concept  or  procedure  was  often  reported 
relevant  in  more  than  one  lesson. 

In  the  opinion  of  the  analysts,  the  training  materials 
(lesson  study  guides  and  learning  supervisors  guides)  are 
of  en ever,  quality  and  often  lack  the  clarity,  specificity. 


and  format  required  for  a  self-instruction  program.  To 
some  extent,  these  weaknesses  are  overcome  by  instructors 
who  clarify  misunderstanding  and  fill  gaps  in  the  training 


material*.  But  to  the  extent  that  instructor  intervention 
is  required,  the  OSC  course  is  not  truly  a  self-instruction 
course. 

First  hand  observation  of  the  conduct  of  the  05C  course 
reveals  that  instructors  frequently  deviate  from  the  format 
specified  in  the  learning  supervisor's  guides.  These  devia¬ 
tions  take  the  form  of  failing  to  cover  content  included 
in  the  Guides  and,  covering  content  not  included  in  the 
Guides.  Compounding  the  problem  is  the  fact  that  instructors 
teaching  the  same  lesson  did  not  always  handle  the  lesson 
in  the  same  manner. 
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APPENDIX  A  . 

05C  ATTRITION  RATES  RELATED  TO  COURSE  INPUTS 


86 


(t*  Cited  In  student  ■nterlal.  Instructor  aaterlal  not  available, 


113-599-1004  IRC)  Install  Radio  Set 

AN/GRC-26D 


91 


Set  10KW 


AD-A128  232 
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